Trio-ER: The Trio System as a Workbench for Entity-Resolution

Entity-resolution (also known as deduplication, record linkage, and reference reconciliation, among others) was one of the original motivating applications [6] for the Trio system, which has been under development at Stanford over the past several years. • Entity-resolution is the process of determining when multiple data records are likely to represent the same real-world entity, and possibly merging such records [5]. • Trio is a research prototype DBMS for managing data that includes uncertainty, and for tracking lineage automatically as queries are performed [2]. Uncertainty is an important component of entity-resolution: match functions may return confidence values, and merged records may retain alternative possible values for attributes. Lineage also is important, since it tracks the original records contributing to a merged result—useful for debugging, and for entity-resolution algorithms that may “unmerge” records. In Trio, lineage also enables computing confidence values for merged records, and it supports useful processing steps in iterative entity-resolution (Section IV-E). Trio-ER is a new variant of the Trio system tailored as a workbench for entity-resolution. The goal of the first version of Trio-ER has been to enable rapid prototyping of an important basic class of entity-resolution algorithms using Trio. As an example of rapid prototyping in Trio-ER, consider a table Hotel(URL,name,zip) containing hotel information. (One of the real data sets we have been using is hotel information from Yahoo! Travel.) Assume we have a string comparison function StrComp(s1,s2) that returns a similarity value in [0, 1]. Suppose we believe two hotel records represent the same hotel if their zip is identical, and they have more than a 0.95 string-match on either URL or name. When records are merged, all variants of URL and name are retained. The following query, expressed in Trio’s SQL extension called TriQL [1], performs one iteration of pairwise matching and merging: